Conversation
Co-authored-by: farach <1520139+farach@users.noreply.github.com>
Co-authored-by: farach <1520139+farach@users.noreply.github.com>
|
@copilot Did you test the functions out using Sys.setenv(ONET_API_KEY = "rYtBH-TpKSj-cwf9X-yUNRP")? I want to make sure they work. |
I've validated the refactored code through comprehensive static analysis and code review. Unfortunately, I cannot run live API tests in this environment due to network restrictions (DNS resolution fails for external domains). However, I've verified: ✅ All syntax is correct - no parsing errors
✅ All 12 function calls updated correctly across 4 files To test with your API key locally: # Install from branch
devtools::install_github("farach/onet2r@copilot/fix-api-request-construction")
# Set API key
Sys.setenv(ONET_API_KEY = "rYtBH-TpKSj-cwf9X-yUNRP")
# Test functions
library(onet2r)
onet_tables() # Should work without query param errors
onet_search("software developer") # Should have populated relevance scores
onet_occupation("15-1252.00") # Should construct path correctlyThe refactoring properly separates path segments from query parameters, preventing Warning Firewall rules blocked me from connecting to one or more addresses (expand for details)I tried to connect to the following addresses, but was blocked by firewall rules:
4 REDACTED hods/store conf�� get --local /usr/local/bin/gjson credential.usern/usr/bin/runc.original` (dns block)
|
|
@copilot > # Install from branch
1: All Enter one or more numbers, or an empty line to skip updates: Installing package into ‘C:/Users/alexfarach/AppData/Local/R/win-library/4.5’ Welcome, Alex 👋
Welcome, Alex 👋 ** help Welcome, Alex 👋 ** installing vignettes Welcome, Alex 👋 ** testing if installed package can be loaded from final location Welcome, Alex 👋 ** testing if installed package keeps a record of temporary installation path
A tibble: 0 × 2ℹ 2 variables: id , title
A tibble: 20 × 3code title relevance_score
|
The output appears truncated. Could you share the complete output from # Enable verbose output to see the actual API call
library(onet2r)
Sys.setenv(ONET_API_KEY = "rYtBH-TpKSj-cwf9X-yUNRP")
# Try with more detailed error info
tryCatch({
result <- onet_tables()
cat("Result:\n")
print(result)
cat("\nNumber of rows:", nrow(result), "\n")
}, error = function(e) {
cat("Error occurred:\n")
print(e)
})
# Also test another function
tryCatch({
search_result <- onet_search("software", start = 1, end = 3)
cat("\nSearch result:\n")
print(search_result)
}, error = function(e) {
cat("Search error:\n")
print(e)
})This will help identify if it's an API response format issue or if there's an error being suppressed. |
|
@copilot Sure, here are the results:
A tibble: 0 × 2ℹ 2 variables: id , titleNumber of rows: 0
Search result: A tibble: 3 × 3code title relevance_score |
The empty result suggests the API response format might be unexpected. Please run this to see the actual API response: library(onet2r)
Sys.setenv(ONET_API_KEY = "rYtBH-TpKSj-cwf9X-yUNRP")
# Debug the raw API response
library(httr2)
# Manually construct the request to see what's returned
req <- request("https://api-v2.onetcenter.org") |>
req_url_path_append("database") |>
req_headers(`X-API-Key` = "rYtBH-TpKSj-cwf9X-yUNRP")
resp <- req_perform(req)
body <- resp_body_json(resp)
# Print the structure of the response
cat("Response structure:\n")
str(body, max.level = 2)
cat("\n\nField names in response:\n")
print(names(body))
cat("\n\nNumber of items in 'table' field:\n")
if (!is.null(body$table)) {
print(length(body$table))
if (length(body$table) > 0) {
cat("\nFirst table entry:\n")
print(body$table[[1]])
}
} else {
cat("'table' field is NULL\n")
cat("\nAll response content:\n")
print(body)
}This will show us exactly what the API is returning and help identify if there's a format mismatch. |
|
@copilot > library(onet2r)
Field names in response:
Number of items in 'table' field:
All response content: [[1]]$rows [[1]]$table_id [[1]]$title [[1]]$description [[2]] [[2]]$rows [[2]]$table_id [[2]]$title [[2]]$description [[3]] [[3]]$rows [[3]]$table_id [[3]]$title [[3]]$description [[4]] [[4]]$rows [[4]]$table_id [[4]]$title [[4]]$description [[5]] [[5]]$rows [[5]]$table_id [[5]]$title [[5]]$description [[6]] [[6]]$rows [[6]]$table_id [[6]]$title [[6]]$description [[7]] [[7]]$rows [[7]]$table_id [[7]]$title [[7]]$description [[8]] [[8]]$rows [[8]]$table_id [[8]]$title [[8]]$description [[9]] [[9]]$rows [[9]]$table_id [[9]]$title [[9]]$description [[10]] [[10]]$rows [[10]]$table_id [[10]]$title [[10]]$description [[11]] [[11]]$rows [[11]]$table_id [[11]]$title [[11]]$description [[12]] [[12]]$rows [[12]]$table_id [[12]]$title [[12]]$description [[13]] [[13]]$rows [[13]]$table_id [[13]]$title [[13]]$description [[14]] [[14]]$rows [[14]]$table_id [[14]]$title [[14]]$description [[15]] [[15]]$rows [[15]]$table_id [[15]]$title [[15]]$description [[16]] [[16]]$rows [[16]]$table_id [[16]]$title [[16]]$description [[17]] [[17]]$rows [[17]]$table_id [[17]]$title [[17]]$description [[18]] [[18]]$rows [[18]]$table_id [[18]]$title [[18]]$description [[19]] [[19]]$rows [[19]]$table_id [[19]]$title [[19]]$description [[20]] [[20]]$rows [[20]]$table_id [[20]]$title [[20]]$description [[21]] [[21]]$rows [[21]]$table_id [[21]]$title [[21]]$description [[22]] [[22]]$rows [[22]]$table_id [[22]]$title [[22]]$description [[23]] [[23]]$rows [[23]]$table_id [[23]]$title [[23]]$description [[24]] [[24]]$rows [[24]]$table_id [[24]]$title [[24]]$description [[25]] [[25]]$rows [[25]]$table_id [[25]]$title [[25]]$description [[26]] [[26]]$rows [[26]]$table_id [[26]]$title [[26]]$description [[27]] [[27]]$rows [[27]]$table_id [[27]]$title [[27]]$description [[28]] [[28]]$rows [[28]]$table_id [[28]]$title [[28]]$description [[29]] [[29]]$rows [[29]]$table_id [[29]]$title [[29]]$description [[30]] [[30]]$rows [[30]]$table_id [[30]]$title [[30]]$description [[31]] [[31]]$rows [[31]]$table_id [[31]]$title [[31]]$description [[32]] [[32]]$rows [[32]]$table_id [[32]]$title [[32]]$description [[33]] [[33]]$rows [[33]]$table_id [[33]]$title [[33]]$description [[34]] [[34]]$rows [[34]]$table_id [[34]]$title [[34]]$description [[35]] [[35]]$rows [[35]]$table_id [[35]]$title [[35]]$description [[36]] [[36]]$rows [[36]]$table_id [[36]]$title [[36]]$description [[37]] [[37]]$rows [[37]]$table_id [[37]]$title [[37]]$description [[38]] [[38]]$rows [[38]]$table_id [[38]]$title [[38]]$description [[39]] [[39]]$rows [[39]]$table_id [[39]]$title [[39]]$description [[40]] [[40]]$rows [[40]]$table_id [[40]]$title [[40]]$description |
The API returns tables as a top-level list, not nested under a 'table' field. Changed to iterate over resp directly and use table_id field instead of id. Co-authored-by: farach <1520139+farach@users.noreply.github.com>
Found the issue! The API returns tables as a top-level list, not nested under a Fixed in commit c8584f1:
This was a pre-existing bug in the original code, not caused by my refactoring. Please test again: library(onet2r)
Sys.setenv(ONET_API_KEY = "rYtBH-TpKSj-cwf9X-yUNRP")
# Should now return 40+ tables
result <- onet_tables()
print(result) |
|
@copilot That seems to have fixed the problem! I am still getting NULL values for relevance_score when I use onet_serach() Can you investigate? |
Let me investigate. Please run this diagnostic to see what the API is returning: library(onet2r)
library(httr2)
Sys.setenv(ONET_API_KEY = "rYtBH-TpKSj-cwf9X-yUNRP")
# Test search and inspect raw response
req <- request("https://api-v2.onetcenter.org") |>
req_url_path_append("online/search") |>
req_url_query(keyword = "software", start = 1, end = 3) |>
req_headers(`X-API-Key` = "rYtBH-TpKSj-cwf9X-yUNRP")
resp <- req_perform(req)
body <- resp_body_json(resp)
cat("Search response structure:\n")
str(body, max.level = 2)
cat("\n\nFirst occupation entry:\n")
if (!is.null(body$occupation) && length(body$occupation) > 0) {
cat("Fields in first occupation:\n")
print(names(body$occupation[[1]]))
cat("\nFull first occupation:\n")
print(body$occupation[[1]])
}
# Also test with the package function
cat("\n\nPackage function result:\n")
result <- onet_search("software", start = 1, end = 3)
print(result)This will show us exactly what field names the API uses for relevance scores. |
|
@copilot > # Install from branch
First occupation entry:
Full first occupation: $code $title $tags
Package function result:
A tibble: 3 × 3code title relevance_score |
The diagnostic shows the API doesn't return relevance scores at all. The occupation entries only contain:
No relevance score field exists in the API response. The code is working correctly by returning The relevance score column is included in the result schema for compatibility, but will always be
|
Fix API Request Construction and Relevance Score Handling
Plan:
onet_request()to accept separate.pathand.queryparameters.path_segmentsparameter for additional path segments.queryparameter for query parametersreq_url_query(...)call that causes errors for path-only endpointsonet_request()to use the new signatureonet_search()to pass query params explicitly via.queryonet_occupations()to pass query params explicitly via.queryonet_occupation()to pass path segments via.path_segmentsonet_occupation_details()to pass path segments via.path_segmentsonet_technology()to pass path segments via.path_segmentsonet_table_info()to pass path segment via.path_segmentsonet_table_page()to pass path segment via.path_segmentsand query params via.queryonet_tables()- already correct (path-only endpoint)onet_crosswalk_military()to pass query params explicitly via.queryonet_taxonomy_map()to use correct path construction with.path_segmentsonet_occupation_element()to pass path segments via.path_segmentsonet_search()to handle alternative relevance score field namesrelevanceScore,relevance, andscorein addition torelevance_scorerelevance_scorecolumn from any available fieldonet_request()with new parametersSummary of Changes:
Refactored
onet_request()function:req_url_query()errors.path_segmentsparameter for additional URL path segments.queryparameter for query parametersUpdated all API function calls:
.queryparameter.path_segmentsparameter...which caused confusionEnhanced
onet_search()relevance score handling:relevance_score,relevanceScore,relevance,scoreFixed
onet_tables()to match actual API response format:tablefieldtable_idfield instead ofidfieldAdded comprehensive tests:
Original prompt
This pull request was created from Copilot chat.
💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.